Which Teams Should Abandon the Run?

Yeah, yeah, yeah. It's a passing league. We got it. And still, according to the numbers, teams aren't passing enough. In the cases of some teams, it's painfully obvious that they should be passing more and running less. As a Ravens fan, I watched another game where nearly every run was simply a wasted down. Most of their paltry positive rushing yards seem to come from trash draw plays on long distances to gain, intended to mitigate very poor field position prior to a punt. It's like they're playing with two or three downs when everyone else gets four.

I wonder if, at some point, when an offense is so much better at passing than running, should it abandon the run almost altogether. On top of the general imbalance in the league, some teams are just throwing away downs when calling conventional run plays. Of course, running and passing generally play off of each other in a game-theory sense. To be successful, passing needs the threat of running, and vice versa. But sometimes, the cost of running is so high for some offenses, that it would be worth the trade-off to forfeit the unpredictability and just pass nearly every down.

It sounds crazy, but take a look at the Expected Points Added per play so far this season (through the 1pm games on Sunday 10/13). The right-most column is the pass-run split. The bigger that number, the greater the imbalance. Pay particular attention to the teams highlighted in red:

Demise of Running?

Longtime reader Eddy Elfenbein wrote me last night about the continuing demise of running in the NFL. From his own site:

The season is still young so this trend may not last, but with two weeks on the books, the number of rushes-per-game is down 4.8% from last year.

But what’s really dramatic is that average yards-per-carry is down by 10.7% (from 4.262 to 3.805).

Combine the two effects, fewer runs going not as far, and the rushing-yards-per-game figure is down 15% from last year. That number has been fairly steady for the last 25 years.

On the passing side, attempts are up 7.8%, completions are up 10.0% and passing yards are up 10.5%. The league-wide passer rating now stands 87.3. In 1994, that would have qualified as fourth-best in the league.

I'm surprised league-wise rushing yards per carry is so low. I don't recall anything below 4.1 YPC, even this early in the season. Aside from that, everything else is part of a continuing trend--something game theory predicts will continue.

Running to Create a 'Manageable' 3rd Down Is Self-Defeating

Note: this is a companion article to last week's column at the Washington Post.

One of the common defenses of a run-heavy offense is that offenses need to make their third downs “manageable,” meaning short enough so that conversion is easier. The thinking goes that if an offense runs on either or both first and second down, it is relatively assured of shorter rather than longer distances on third down. At first look, this makes a lot of sense. After all, who wants to face third and long?

There are two problems with this argument. The first is that football is not a game of piling up first downs. The days of inching toward the goal line on 18-play drives are long gone if they ever existed at all. Football is, for the most part, a game of maximizing score differential, and the concept of Expected Points shows that NFL offenses are generally running too often on first and second down.

The second problem is that even if gaining a first down is the primary objective, running on first down is becoming a worse idea every year. The graph below shows that passing on first down leads to a conversion more often than running on first down. As usual, I limited the data to plays in ‘normal’ football situations, when the score is relatively close and time is not yet a factor at the end of either half.

Washington Post: 'Manageable Third Downs'

This week's article at the Post looks at how third down conversion rates can mislead coaches into poor strategic approaches.

Offenses are better off thinking of their three downs (and fourth when the situation requires) as isolated opportunities for ten-yard conversions rather than stepping stones toward what coaches call a “manageable third down.” The best third down situation isn’t third and 1 or even third and inches. It’s converting on first or second down, before ever reaching third down. Rather than seeking a short third down situation, offenses should be avoiding third downs whenever possible.

Expect Even More Passing Yards, and Why It Matters

Remember the passing explosion to start the NFL season last year? Get ready for even more. 2011 was not a one-year blip but instead was part of an accelerating trend toward more potent and more frequent passing. This isn't statistical trivia either, as this trend has dramatic implications for how the game should be played.

Take a look at Adjusted Net Yards Per Attempt (courtesy of PFR), which with just one number incorporates passing efficiency, interceptions, and sacks. Since the dawn of the modern passing era, passing has become steadily more lucrative. But since 2004, the rate of increase in average ANY/A has accelerated. The 2011 season featured the most successful passing game ever.

For context, compare the graph above with the next one. This shows the same trend but for rushing yards per carry. There is a very shallow increasing trend since a trough in the mid 1990s, but it pales in comparison. The jump in net passing last year alone is larger than the increase in rushing over the entire period. (I've kept the scales of both graphs identical for a pure comparison.)

Weather Effects on Passing

My last post looked at the effect of temperature on home field advantage. We saw that cold weather put dome and warm climate teams at a disadvantage. The post was titled How Does Temperature Affect Road Teams?, but I really didn't answer that question. I measured the size of the effect, but I didn't solve the riddle of actually how temperature makes a difference. This post will begin to look at just how weather makes a difference, starting with the passing game.

Here's how passing fares for home and visiting teams by temperature. The chart below shows  Adjusted net Yards Per Attempt (AYPA), which accounts for sacks and interceptions, according to temperature. Keep in mind there are smaller sample sizes at the extremes.

Brees versus Marino, 2011 versus 1984

On Monday night we saw history. Drew Brees eclipsed Dan Marino's record of 5,084 total passing yards on a late-game touchdown pass to Darren Sproles. Regular readers know that I'm no fan of most ever or least ever records because they're usually just trivia that end up giving the word 'stats' a bad name. Readers also know that total passing yards is not a particularly meaningful way of measuring a quarterback's skill. But it's hard to let the occasion pass without taking note.

The Monday Night Football crew did a good job of reminding viewers of the context of the record, even going as far as providing analysis showing how far above average both Marino's and Brees' were for their respective seasons. The NFL's passing numbers have steadily inflated over the years, largely due to rule changes that favor offenses. But like many other sports, it's possible that the athletes have simply improved over time. Defenders can improve too, but who's to say that athletic improvement on both sides of the ball doesn't disproportionately favor the offense. The fact is we simply have no way to tell.

We can use some statistical tools to get a feel for how outstanding each season was. Drawing the line at the top 30 passers in both seasons, we can calculate the number of standard deviations Marino and Brees stand above the season average. Marino's 1984 was 2.4 standard deviations above average, while Brees' 2011 (so far) is 1.9 standard deviations above average. Marino achieved his numbers on 564 attempts while Brees has 622 attempts, and counting. Brees has 13 interceptions compared to Marino's 17. According to PFR's Adjusted Net Yards Per Attempt, which factors in yards, attempts, interceptions, and touchdown passes, Marino beats Brees 8.9 to 8.0.

One of the unspoken assumptions when discussing Marino's 1984 record is that his record is a 'pure' or 'true' record, and the record set in 2011 is asterisked by the liberal passing rules of today's NFL. But do you know who's record he broke and when that was set? It was Dan Fouts in 1981. Before that the record belonged to...Fouts in 1980. And before that...Fouts in 1979. But prior that, the pro record was an AFL mark set by Joe Namath in 1967.

Identity Crisis

If there’s one meaningless word thrown around by football analysts in the last couple years, it has to be identity. “The Jets offense has an indentity crisis.” “The Ravens need to find their identity on offense.” “The Eagles have lost their offensive identity.”

Are we talking about professional football teams or teenagers trying to figure out whether to hang out with the jocks, dweebs, preppies, or wastoids?

Identity has replaced rhythm as the most meaningless word in the NFL. Remember those days when every broadcaster at one point in the game had to say, “The 49ers need a couple of completions here to get into a rhythm"?

What the hell does that even mean?

Deadspin/Slate Roundtable: Passing in 2011

Here's a second post in the roundtable series at Slate and Deadspin. This one encapsulates and updates my recent look at the causes of the passing explosion in 2011. It's a little more readable and entertaining than the original graph-filled post.

What Happened to the First Round RB?

In the five-year period between 1970 through 1974, running backs made up 20% of all first round NFL draft picks. That's one out of every five. As recently as the 1985-1989 period, RBs made up 19% of first rounders. But by the most recent decade, from 2000 through 2010, RB selection was cut in half--down to about 10%. Last night, only 1 of the 32 players chosen (about 3%) was a RB, and he was chosen 28th, near the bottom of the round.

The graph below illustrates the trends in how teams favor each position over the past 41 years. Most positions are fairly stable. Click to expand.

Tables vs. Graphs

I'm not sure about everyone else, but I've got a very visual brain. I'm one of those guys at work who can't have a conversation without going to the whiteboard, if only to organize my own thoughts. I don't think I'm alone, either. Many theorists believe one reason humans became such smart monkeys is that we co-opted the huge visual-spatial part of our brains to use for abstract thought.

The concept of time is one example. We think and talk of time, a concept virtually without its own terminology, in terms of space and motion: Time goes by...Our best days are ahead of us...I'm looking forward to next season. The blathering talking heads on CNBC can't go 20 seconds without convulsively saying the phrase going forward whenever referring to the future.

Abstract sports concepts like win probability are no exception. We would all call 2-yard run on 4th and 1 a big play, even though it was anything but literally big. How would we characterize a 38-35 game? As a high-scoring game, of course. We are universally comfortable speaking about abstract concepts in terms of the metaphors of physical position, size and motion, and it's a window into how we think. That's why I'll take a graph over a table of numbers any day.

Plotting Run vs. Pass Success Rates (And How to Beat NE)

It might be a little late in the season for stuff like this, but better late than never. Often we can learn a lot by taking information we already have and graphing it. In this case, I've taken each team's Success Rate (SR) stats for running and passing and plotted them against each other.

As you might recall from a post early in the season, SR is one stat where passing and running actually correlate, suggesting that "success" is the component of performance that coaches try to optimize. Simply put, from the perspective of game theory, it's how we can tell that running "sets up the pass" and vice versa. SR for both running and passing is also a reliable predictor of team success, i.e. winning.

If we plot each team's pass SR by its run SR for 2010, we should expect to see a correlation, represented by a diagonal trend. The better teams will be on the top and right, while the worse teams will be on the bottom left. Also, we would not expect to see many teams that are very good in one aspect, either running or passing, but not any good in the other.

Almost Always Go for 2-Point Conversions?

In the Buccaneers-Redskins game this past Sunday, the Redskins were able to score a potentially game-tying touchdown at the end of regulation, only to fail to hit the extra point due to a mishandled snap. Gregg Easterbrook suggested the Redskins should have gone for the two-point conversion, which is a plausible strategy in many circumstances. But Easterbrook went on to add this little tidbit: "Rushing deuce attempts are about 65 percent successful in the NFL -- a better proposition than the 50/50 of advancing to overtime."

It's well established that 2-point conversion attempts are successful slightly less than 50% of the time, so could the 65% number for runs possibly be true? If so, what would that mean for NFL strategy?

There have been 718 2-point conversion attempts from 2000-2009, including playoff games. Overall, they've been successful 46.3% of the time. But this is slightly misleading because it includes aborted kick attempts. If we weed those out, along with some other mysterious plays, such as Josh McCown's kneel-down while trailing by 5 points in the final few seconds of the Cardinals-Vikings 2003 game, we get a different answer. For all normal 2-point conversions, the success rate is 47.9%.

Now look at the success rate broken out by play type:

Deep vs. Short Passes

Over the past couple years, we've learned that passing well is  more important than running well in terms of winning games. We've learned that passing has become more and more lucrative over the years. And we've learned that offenses should pass more often, particularly outside the red zone and on 1st down. But 'passing' is a large category, encompassing everything from a screen pass 2 yards behind the line of scrimmage to a 50 yard bomb. 

Beginning in 2006, the NFL classified every pass attempt as either 'short' or 'deep,' where deep means anything past 15 yards. About 19% of pass attempts are classified as deep. Unfortunately, that's all we get, so we can't tell a screen from a 14-yard down-field pass attempt. Still, it allows us to begin to pull apart different types of passes and examine them in one more layer of detail.

In normal football situations, in which the clock is not yet a factor and the score is relatively close, pass plays, including sacks, yield an average of +0.08 Expected Points Added (EPA), while run plays yield an average of +0.01 EPA. Which type of pass is more responsible for that advantage, risky deep passes or safer short passes?

What Is the Break-Even Run Success Rate?

I've been looking at run Success Rate (SR) lately, and it appears to be a fairly important indicator of team success. But if I said that a particular team's run SR was 45%, how would you know if that's any good? It's less than 50%, so does that mean it's bad? So I decided to plot Expected Points Added (EPA) against SR to find out where the break-even point is.

When I plot run EPA per play vs. run SR (for 2000 through 2009), the break-even point is where the best-fit line crosses EPA/P axis--just above 41%.

How Coaches Think: Run Success Rate

Before tools such as EPA and WPA were available, I relied on team efficiency stats to estimate team strength. Yards per pass attempt or per run attempt worked out to be very good estimators of how good a team was, especially if ‘good’ is defined as being likely to win forthcoming games. Efficiency stats had the added benefit of being relatively simple, widely available, and easy to calculate.

Efficiency stats also worked well in regression analysis. In a regression model, it’s best if the predictor variables are independent of each other. In other words, the less each predictor variable correlates with the others, the more valid and reliable the resulting model will be. Passing and running efficiencies in the NFL correlate weakly. Over the past 10 seasons, offensive passing and running efficiencies for each team correlate at 0.09 (where 1 would mean lock-step correlation and 0 would mean complete independence.)

Passing = Winning

Advanced NFL Stats owes its start to an old water cooler debate: What's more important, offense or defense? Running or passing? A few years ago, I still had some statistical software left over from grad school loaded on my laptop, so I thought, "Hey, maybe these are questions that can be definitively answered." I tried to answer those questions with one of my original posts three years ago, What Makes Teams Win. When I read my older stuff, I sometimes want to cringe, but not with that one. It holds up very well, and it's well worth revisiting for newer readers, this time with more data. In this post, I'll do just that, focusing on the relative importance of running and passing.

When I was little, my dad taught me the inanity of the 'running leads to winning' fallacy. We'd watch a game on Sunday, and invariably we'd hear the announcers talk about how a team always wins when their star RB got at least 25 carries or so. They'd wax poetic about the noble nature of pure, old-fashioned, run-it-up-the-gut football. My dad would say, "Yeah, by that logic, teams should start kneeling in the first quarter. Kneeling leads to winning, right?"

Run-Pass Balance--A Historical Analysis

I’ve been writing a lot about run-pass balance lately, and part of my theory of why teams are perhaps passing less often than they should has to do with the evolution of the sport. Rule changes over the recent decades have generally favored passing. Changes in pass blocking rules and in pass interference rules have made it easier to pass the ball successfully. Even subtle rule changes such as the definition of possession and “control” may have made receiver fumbles less likely.

Tactics and play selection have been refined over the years to take advantage of the rule changes, but I’m not sure that they’ve completely caught up. Results from several studies, including my own, have suggested that in most situations, passing is more lucrative than running. This imbalance implies that passing should be selected more often. As defenses respond to expect more frequent passing, the payoff for passes will decrease as the payoff for runs increases. Eventually, there is an equilibrium where the payoffs should be equal.

In this post, I’m going to look at very simple historical trends. As you’ll see in the graphs below, there is evidence that the current run-pass balance has not responded fully to recent increases in the payoff of passes. All data come from PFR's very cool league historical pages.

Run-Pass Imbalance In the Red Zone--1st Downs

Just before halftime in last year's Super Bowl, on first and goal from the one, Kurt Warner threw the ball directly into the arms of James Harrison who rumbled 100 yards for a touchdown. With so little time left in the half, passing was the obvious call, but that play highlights the dangers of passing so close to the goal line.

Game theory tells us that when payoffs for strategies are unequal, the strategy with the higher payoff should be chosen more often. We've seen that between the 20 yard lines payoffs for passes are consistently higher than for runs on 1st down, but inside the 20 running becomes more lucrative. Now let's take a look at the red zone in more detail, where the stakes get higher and the field gets shorter. On 1st downs in the red zone, should offenses run or pass more often, or do they already strike the right balance?

Run-Pass Imbalance on 2nd and 3rd Downs

I've recently been looking at the imbalance in the payoffs for running and passing on first downs. The results suggested that most teams should generally pass more often outside the red zone and run more often inside the 10-yard line. What about 2nd and 3rd downs?

Game theory tells us that when the payoffs for two strategy options are unequal, the strategy option with the higher payoff should be selected more often. As the opponent adjusts to counter the new mix of strategies, the payoff of the favored option will decline while the unfavored option becomes more lucrative. Eventually, the payoffs for both options equalize, and at this point the overall payoffs are optimum. In two-player zero-sum games this is known as the minimax, or more generally as the Nash Equilibrium.

I used Expected Points (EP) to value the payoff of each play. Expected Points measures the net point advantage that the play result gives to an offense. It captures the value of yardage gained and lost, first downs, sacks, penalties, turnovers, and everything else in terms of equivalent point value. The change in EP resulting from a play is called Expected Points Added (EPA).

One of the things EP does not measure is the time value of a play. In situations when a team has a significant lead, the true value of a run includes the time burned off the clock. To a team behind late in a game, pass attempts have more value because they are more likely to stop the clock. For this reason I only include plays in the first and third quarters and when the score is within 10 points. This excludes trash-time plays and plays affected by the clock.

Run-Pass Imbalance by Year

I've been chronicling the imbalance in the payoffs of runs and passes on first down in several recent articles. One of the possible explanations for the imbalance is that coaches are too slow to adapt to the NFL rules that seem to become friendlier to the pass year after year. If so, maybe the adaptation to the new realities can be seen in a decrease in the payoff imbalance over recent years.

The graph below charts the difference in payoffs between passes and runs by year. As with my previous posts, data are limited to 'normal' football situations--when the score is close and when time is not yet a factor. As you can see, there may be a slight decreasing trend in the imbalance, suggesting coaches might be catching up.

2009 Team-Specific Run-Pass Balance

Recently I've been looking at run-pass balance on first downs based on a principle of game theory. When strategy mixes are optimized, the two strategies will ideally produce equal payoffs. If they aren't equal, then the better strategy should be selected until the opponent responds with his own counter-strategy. Results suggested that, in the NFL overall, the gains by passing on first down exceed those by running. In turn, this suggests that offenses should pass more often than they currently do.

However, every team has its own relative ability between passing and running. You can't just tell the 2009 Raiders to start passing more often. Their running game may actually be superior to their passing game in terms of expected payoffs, so while most teams should be passing more frequently, it's possible a minority of teams should be running more often.

(Much) More on 1st and 10 Run-Pass Balance

In a recent article I presented evidence that offenses should generally pass more often on first down. Accounting for both the potential gains and the potential risks of each type of play, passing tends to lead to a greater net point advantage than does running. The analysis was based on a concept known as Expected Points, which measures the average point advantage an offense can expect given a down-distance-field position situation.

Expected Points incorporates all the various factors such as turnovers, yardage gains, sacks, incompletions, first down conversions, scoring, and so on. But I thought it would be helpful to dig deeper to investigate how and why passing appears more advantageous. In this article, I'll present a series of graphs comparing running and passing on first down, each one looking at a different facet of the game.

Offenses Run Too Often On 1st Down

NFL offenses generally run too often on 1st down. Accounting for the relative gains of each play type, and accounting for the risks of turnovers, offenses should pass more. There is currently an imbalance, where teams are too often running directly into defenses that are expecting runs.

Game theory tells us that when there are two strategy options, like run and pass, the expected payoffs for both options should be equal. You really don't need game theory to intuitively understand this. If one option yields a better payoff, then it should be chosen until the opponent responds with a strategy change of his own. Eventually, as the opponent responds, the payoffs for the two options equalize. The point at which the strategy mix equalizes payoffs is known as the minimax, or sometimes called the Nash equilibrium. The resulting strategy mix, or run-pass balance in this case, produces the best overall, long-run payoff.

When there are two strategy options and one of them yields a much higher payoff, it tells us two things. In this case, passing is more lucrative than running on 1st down, and this tells us: 1) offenses should be passing more often, and 2) for now, defenses should continue to be more biased toward stopping the run.

Full Review of Game Theory Run-Pass Balance Study

A new paper on game theory and run-pass balance in the NFL, Professionals Do Not Play Minimax: Evidence from Major League Baseball and the National Football League, says that offenses run too often and play calling is too predictable. The authors, Kenneth Kovash and Steven Levitt, construct a success metric to value the outcome of NFL runs and passes from the 2002-2005 seasons. Then using regression models, they estimate and compare the values of a typical run and a typical pass. They also construct a regression to test if play calls can be predicted to any degree based on the previous play call.

Game Theory

Game theory tells us that in a 2-player zero-sum game, if both players are playing the optimum mix of strategies, the long-term average payoff from each strategy will be the same. If you have two general strategy options, like run or pass, you can’t just choose one of them all the time. That would make the defense’s job pretty easy. So you need some sort of unpredictable mix of strategies. The question is, what’s the optimum ratio?

A New Academic Study on Game Theory and Run-Pass Balance

There’s a new study on run-pass balance based on game theory minimax equilibrium. The study is called Professionals Do Not Play Minimax: Evidence from Major League Baseball and the National Football League and it’s from Kenneth Kovash and Steven Levitt (of Freakonomics fame).

The authors created their own version of  Expected Points as their measure of play success. Using a giant regression model that accounts for all sorts of confounding variables, they find passes lead to more success than runs. Game theory would say that, ideally, both strategies should yield the same amount of success.

Thoughts on the Apparent Unimportance of Run Defense

John Morgan of the Seahawks blog Fieldgulls.com asked me recently about why run defense appears relatively less important in terms of winning than do other facets of the game. John writes:

You state defensive rushing yards per attempt allowed is the least important component to winning, but I wonder if that factors in game situation. Losing teams are likely to reduce their yards per attempt allowed when winning teams are running out the clock.

John makes a good point, and indeed the team-stat regression model I used when I made that conclusion did not take game situation into account. John goes on to point out that teams that are already behind may face a large number of predictable run-out-the-clock runs, which would make their run defense appear better. Plus, the game theory aspects of running and passing should enable a good run defense to make stopping the pass easier.

I think this is an interesting point, and I soon should be able to test how much late-game runs affect a team’s overall defensive efficiency after some improvements to my play-by-play database. A preliminary look indicates it probably doesn’t make much of a difference. Still, I think John raises good points about the game theory aspect and predictability.

In theory, good a run defense should make a pass defense better. And the stats suggest it does. The correlation between defensive running efficiency and passing efficiency is 0.20. Some of that correlation has to do with the fact that superior defensive athletes are superior against both the run and the pass. But some of it should also be due to the game theory aspect.

(In comparison, offensive running and passing efficiency correlate at 0.13. I think the difference is that there is a lot more variance in offensive passing than in other facets due to the focus on a single player’s ability. The quarterback’s contribution is so critical to passing in ways that aren’t applicable to the other aspects.)

According to game theory principals, defensive running and passing would only be equally important if offenses and defenses were operating at the game equilibrium point-- that is, they’re playing the optimum mix of passing and running. But I think they might not be.

My theory is that this may be why run defense appears so unimportant. Say teams are not operating at the game equilibrium, and passing is, on balance, a more lucrative strategy than running. In other words, there really is a considerable passing premium where the payoff for a pass is generally higher than a run, all things considered.

Having a good run defense would therefore be somewhat self-defeating. Take the 2007 Vikings defense that gave up only 3.1 yards per run (good) but 7.0 yards per pass (bad). Facing such a solid run defense, a good offensive coordinator is forced to pass more…which would be a far more effective thing to do in the first place, especially against a relatively weak pass defense. A team like the 2007 Vikings would essentially be forcing their opponent to unwittingly play a more efficient and effective strategy, all the while exploiting their own weakness.

Passing Predictability Part 2

Here's a tip: If you author a sports research website, never start an article as "Part 1" unless you already have a Part 2 ready to go. Way back at the end of March I began looking at how predictability affects passing success. I looked at "10 yards to go" situations--1st and 10, 2nd and 10, and 3rd and 10 plays. Passes are least predictable on 1st down and most predictable on 3rd and long. With 10 yards to go, passes are called roughly half the time on 1st and 2nd down, but on 3rd and 10 passes are called 91% of the time.

Since then, I've been tinkering with my NBA and NHL in-game win probability sites. This post will officially tie the loose end that's been dangling since the NCAA Tournament.

In Part 1 of this article, we saw the average gain in each of these situations:

Yds Per Attempt by Down, 10 Yds To Go

Type	1st	2nd	3rd	Total
Pass	7.0	6.3	6.5	6.9
Run	4.2	4.4	6.9	4.3
Total	5.5	5.4	6.5	5.6

But there were several wrinkles in this analysis. First, there are interceptions to consider. Interceptions become much more probable on 3rd and 10.

Interception % by Down, 10 Yds To Go

	1st	2nd	3rd	Total
Int Rate	2.6	2.9	3.5	2.7

Combining Yards per Attempt and Interception rate yields Adjusted YPA, which is YPA with a -45 yd adjustment for every interception thrown.

Adj Yds Per Attempt by Down, 10 Yds To Go

Type	1st	2nd	3rd	Total
Pass	5.9	5.0	4.9	5.6
Run	4.2	4.4	6.9	4.3

But there is still a problem. Adj YPA underestimates the drop off from 1st to 3rd down in passing effectiveness because defenses will allow gains, as long as they're not more than 9 yards. So we have to limit our observation to say the reduction in passing effectiveness due to predictability is likely at least 1 full adjusted yard per attempt.

Except that there's still another problem. There's a bias in the data because poor passing teams will face more 2nd & 10 and 3rd & 10 situations. So the 2nd and 3rd down numbers are lower than would be representative of the league as a whole. In other words, poor passing teams 'get more votes' in the analysis.

The solution is to give each team (or each team-year, actually) equal votes. Instead of averaging the gains for all 2nd and 10s for the entire league as a whole, I first averaged them by team-year, then averaged them by team.

Broken Out by Team & Year, Re-Averaged

Stat	1st	2nd	3rd	Total
YPA	7.0	6.3	6.5	6.8
Int Rate	2.6	2.9	3.5	2.7
Adj YPA	5.8	5.0	4.9	5.6

As it turns out, the numbers are nearly (but not completely) identical using this method. The effect of the bias isn't very pronounced and we're left with the original estimate. Going from about 50% predictability to 90% predictability costs at least 1.0 Adj YPA.

Another notable observation is that although 1st &10 and 2nd & 10 have about the same run/pass balance of about 50%, passes on 1st and 10 are considerably more effective--5.8 vs 5.0 Adj YPA. I'd have to think that the difference may be largely due to play action.

Passing Predictability Part 1

In its simplest form, play calling in football is a game of two strategy choices--pass or run. Game theory teaches us that the mix between the two strategies needs to be unpredictable to be effective. Ideally, there should be an optimum ratio of running and passing based on the expected benefits and risks in any given situation. In this post, I'll examine the effect of predictability by comparing passing success when passes are most and least predictable.

In reality there are several different variations of runs and passes an offense can choose. And defenses don't have discrete strategy choices at all. They can select from a continuous range of strategy bias from run certainty to pass certainty. But for now, I'll limit my discussion to just the two most basic options.

Refresher

Consider the graph below from an earlier post on game theory in football. The horizontal axis of the graph represents a range of strategy choices for an offense. On the left is "always run," and on the right is "always pass." Between each extreme is a range of mixed strategies. Halfway between would be a balanced 50/50 run-pass mix.

The colored lines represent the utility to the offense of each of its strategy mixes when the defense chooses each of its own strategies. In this example, if the offense were to always run (left edge of the chart) and the defense always played a pass defense(red), the offense would be very successful. And if the defense always played run defense (blue), the offense would certainly suffer.
Usually, the strategy mixes settle at an equilibrium. In the example above, the best the offense can do is a mixed strategy of running 63% of the time and passing 37% of the time. In this situation the defense should never blitz because the utility to the offense at the 63/37 mix is higher against a blitz than either other defensive strategy.

Keep in mind that yards do not equate with utility. Utility in football is ultimately win probability. The outcome of every play either increases or decreases a team's chances of winning. There are other considerations beyond yardage gained. For example, a 3 yard gain on 2nd and 2 may be better (higher utility) than a 4 yard gain on 2nd and 6. The risk of a turnover also has to be considered. But again, the analysis is always dependent on the situation and always assumes unpredictability. The benefit of the optimum strategy mix is lost if an offense repeatedly calls run-run-pass all game.

The Goal

What I really want to do is replace that example graph with a real one, based on real numbers. That might be something very useful. We could know truly optimum play calling mixes, or how to capitalize on opponent non-optimization. In a way, we could solve the 'game within a game' of play calling.

This is a Quixotic task, I realize. In fact, it sounds quite kooky. But we could narrow the focus enough and eventually begin to get some idea of what the utility graph looks like in certain situations. What we'd really need to do is nail down some of the critical values in the graph, such the endpoints of the utility lines at 100% run or 100% pass, or where the equilibrium point is.

Analysis

I decided to look at "10 yards to go" situations--1st and 10, 2nd and 10, and 3rd and 10 plays. When is passing least predictable? On 1st down. When is it most predictable? On 3rd and long.

All the numbers that follow are from all 10-yards-to-go scrimmage plays in the first 3 quarters of regular season games from 2000-2007. The only other limitation was that the game score was within 10 points. I wanted to exclude situations when teams exercised an abundance of either risk or caution.

Note the percentage of play types called on 1st, 2nd, and 3rd downs (with 10 yards to go). There is a fairly even split between run and pass calls on 1st and 2nd downs. On 3rd and 10, the a pass is far more expected.

% of Play Types by Down, 10 Yds To Go

Type	1st	2nd	3rd	Total
Pass	47.2	52.7	91.1	49.6
Run	52.8	47.3	8.9	50.4

Although 91.1% isn't 100%, it's close to where the anchor point on the lower right side of the game theory graph--almost the pure pass vs. pass defense strategy combination. Now let's look at the average outcomes for these situations.

Yds Per Attempt by Down, 10 Yds To Go

Type	1st	2nd	3rd	Total
Pass	7.0	6.3	6.5	6.9
Run	4.2	4.4	6.9	4.3
Total	5.5	5.4	6.5	5.6

When passing is most predictable, it yields half a yard less than on first down, when it is less expected. Conversely, running is most successful when it is least expected.

At this point, I should point out that passing on 3rd and 10 yields slightly more yards than on 2nd and 10, which isn't completely what we'd expect. This is almost certainly because defenses will allow short complete passes on 3rd down in exchange for being relatively assured to be able to stop the gain short of 10 yards. This is part of the problem posed by the fact that yards does not equal utility. We'll have to dig a little deeper. The next table lists interception rate by down.

Interception % by Down, 10 Yds To Go

	1st	2nd	3rd	Total
Int Rate	2.6	2.9	3.5	2.7

Now we see more what we'd expect--a slight increase from 1st to 2nd down, then a large jump on 3rd down, in accordance with the associated increases in passing predictability. The next table lists adjusted yards per attempt, which is YPA with a -45 yd adjustment for every interception thrown. Adj YPA, however, still exhibits the same problem as plain YPA. It underestimates the drop off from 1st to 3rd down in passing effectiveness because defenses will allow gains, as long as they're not more than 9 yards.

Adj Yds Per Attempt by Down, 10 Yds To Go

Type	1st	2nd	3rd	Total
Pass	5.9	5.0	4.9	5.6
Run	4.2	4.4	6.9	4.3

So what we can say is, the reduction in passing effectiveness due to predictability is likely at least 1 full adjusted yard per attempt. The drop from 1st down to 2nd was 0.9 yards, so the true reduction in effectiveness from 1st to 3rd down may be far larger.

Except that there's a problem with this analysis. There's a bias in the data. Which teams are more likely to face a lot of 2nd and 10s and 3rd and 10s? The ones that stink at passing. So the 2nd and 3rd down numbers are lower than would be representative of the league as a whole. In other words, poor passing teams 'get more votes' in the analysis.

Fortunately, I think I've solved this problem. I'll explain it in the second part of this article.

Establishing the Run? 2

A few days ago I posted a look at whether run gains increased the more often a team runs. In other words, does running more frequently wear down a defense and allow longer runs later in the game?

My analysis was pretty straight-forward. I numbered each team's run in each game, then plotted the average gain of every first run, second run,...20th run in a game. There was no increase in the average gain with later runs, and so I took this as evidence that frequent runs do not fatigue defenses and contribute to longer gains. What I should have said is I failed to find evidence to that effect.

Commenters had a lot of pushback on my conclusions, and for good reason. Among the criticisms were:

1. Late-game meaningless runs to run out the clock by teams with large leads will tend to be short because defenses expect runs.
2. Late-game runs within field goal range might be short for similar reasons. If a team only needs 3 points to tie or win, they'll almost always run the ball to avoid an interception, and defenses know this.
3. Average gain doesn't tell the whole story. It's helpful, but it's just one number.

So I did a few more things. I limited the data to runs when the score was within 8 points--a single score. I also limited the data to runs from outside field goal range, defined as the 34 yard line.

Also, instead of looking at average gains according to which run it was for each team in a game, I created a histogram. For those not familiar, it's a frequency plot of how often runs go for each amount of yardage. Knowing the average is good, and median might be better, but the full distribution can tell us much more of the whole story.

I grouped the runs into two sets of interest. The first set is each team's first 15 runs of a game, and the second set is for each team's 30th and later runs. If there are noticeably more instances of longer gains for the later runs, then we can say we have evidence for the run-fatigue theory.

Data is from league-wide regular season runs from 2000 through 2007.

There are far games with run attempts of 30 or more than games with up to 15 attempts, so I normalized the distributions. Each distribution is plotted as a percentage of the total in each group. For example, we'd read the graph by saying about 13% of runs 1-15 are for 2 yards. The same is true for runs 30+.



I won't make any inferences--for now. I'll just make some observations and let others draw their own conclusions. Please share your thoughts with comments.

The distributions are nearly identical, except in two places. The first place is kind of a quirk just short of 10 yards of gain. (There is an anomaly of NFL stat-keeping at 10 yards. If the ball passes 3 yard markers, it's a 3-yard gain. If it passes 4, it's a 4-yard gain, and so on, except for when the ball is within a yard of a 1st down. On 1st and 10, you can pass 10 yard markers and still only be credited with a 9-yard gain if the nose of the ball doesn't get past the first-down marker. Since 10 yards to go is by far the most common to go distance, the effect of the anomaly will be most pronounced short of 10-yard gains, making 9-yard gains more common.) Because there are far fewer late runs than early runs, the late runs are statistically more susceptible to this quirk.

The second difference is more relevant. Since there are so few gains beyond 25 yards, I grouped gains beyond that distance together. That's where the little up-tick is at the right end of the graph. Additionally, keep in mind that many, if not most, very long runs are touchdowns and are truncated by the goal line. An 70-yard run from an offense's own 30 could very well have been a 90-yard run had the line of scrimmage been a the 10 yard line.

There's a very slight advantage for the late runs. It's statistically significant, but extremely small--about 2.2% vs. 1.8% of runs beyond 25 yards. The significance, however, is possible only because the sample sizes are huge (n=33,952 for early runs and n=2,536 for late runs).

Establishing the Run?

"Establish the run" might be the three most over-used words in football analysis. Bad football analysis, that is. You would give yourself one heck of hangover playing one of those fraternity-type drinking games going bottoms-up every time you heard that phrase on a Sunday.

Establishing the run could mean a lot of different things. To most people I think it means that offenses will demonstrate the willingness to run frequently, hoping that defenses will bias towards stopping run later in the game. I think this interpretation has been debunked fairly thoroughly, so I'm going to look at it from a different angle.

Establishing the run could mean running often early in a game in hopes that a defense will weaken, and runs will be longer later in the game. I think this is more plausible theory. It makes sense on the surface. Running plays tend to batter the defense while passing plays allow the defense to go on the attack. By the fourth quarter, it's believable that defenders would be battered and fatigued. Defenders would get off of blocks slower and tackles would be sloppier. Watching Baltimore's two consecutive long TD runs late in the fourth quarter at Dallas in week 16 made me wonder: Does running frequently lead to longer gains?

To answer the question, I compared the average gains from running plays based on when they took place in a game. By "when," I mean which run play it was, not what time it took place in the game. In other words, if running frequently fatigues a defense, then the gain of a team's 30th run should tend to be longer than its first run.

We'd expect that the more runs an offense calls, the longer the subsequent runs should tend to be. The graph below depicts the average gain of each rushing play based on its order in which it was run. It plots the average gain of each team's first run of a game, 2nd run, 3rd run, ...etc.

(Data is from all regular season games from 2000-2007. All runs except kneel-downs were included.)

There's no increase in average gain as the number of runs increase. A team's very first run of a game is just as long as, if not longer than, the 20th, 30th or even 40th.

This result is despite the expectation that teams that are good at running would naturally run more often. Teams that are ahead toward the end of the game Those teams would therefore be the ones that we'd expect would accumulate more attempts. If so, we'd see the average gains increase as the run attempts mount. But we don't.

So this is evidence that runs are just runs, no matter how many have come before them. An offense can expect the same average gain on the first snap of the game as on the 80th, after 30 previous run plays.

The run may not set up the run, but does it set up the pass? Does running frequently allow longer gains on passes later in the game? I'll look at that question next.

Edit: Follow-up here that addresses many of the comments below.

Predictability

Over the past few months I've been writing about how game theory can help us understand play-calling in football. Not only can it help us understand why coaches call the plays they do, but it can instruct us on what the optimum balance of play types should truly be. Offenses always need a mix of strategies to maximize their gains, no matter how much better they might be at running over passing or vice versa. But just important as the ratio of the strategies is the unpredictability of each call.

The mix of plays needs to be random to be effective. That's not to say play calling should be picked willy-nilly out of a hat. For every situation there will be an optimum ratio of play types. For example, on 3rd and 1 situations, teams should generally run at least about 85% of the time. But within that 85/15 run-pass mix, the decision needs to be unpredictable, which means it must be random and independent of the previous play call. The problem for play callers, and the opportunity for defensive coordinators, is that people are terrible at randomizing.

There's a story about a statistics professor who challenges his class to a contest. He divides the class in half and tells one group to flip a coin 100 times and write the sequence on the board-- THHTTH... The other group is told to invent and write their own sequence of heads and tails on the board as randomly as they can, without looking at the other group's sequence. The professor says that if he can't tell the true random sequence from the fake one, he'll give everyone an A (or something). He leaves the room until both groups are done, then returns and instantly spots the fake sequence.

The professor can identify the fake random sequence so easily because it has too many alternations between heads and tails, and too few long streaks. The fake sequence looks like HTTHTHHTHT, while true randomness often looks like HHHHHTHTTH. True randomness can be quite streaky (which is partly why people fall for fallacies like "being in the zone" or "the hot hand").

If I'm a defensive coordinator, I'd like to know what kind of play the offense is going to run. I don't need absolute certainty--any idea is better than no idea. For example, for all 2nd and 10 plays in the NFL, offenses run the ball 46% of the time. But what if defensive coordinators could know that based on other circumstances, this particular 2nd and 1o will be a pass 80% of the time?

Take a look at run-pass balance on 2nd down situations. The graph below shows the percentage of run plays on 2nd downs according to the yards-to-go situation. There is one noticeable aberration at 2nd and 10: runs are far more frequent.


Why would runs be far more frequent on 2nd and 10 yards to go than on 9 or 11 yards to go? The task facing the offense is not meaningfully different. What makes 10 yards to go so special?

The key is that 2nd and 10 situations are several times more likely to occur due to an incomplete pass than due to a run for no gain. This suggests that, effectively, teams are significantly more likely to run following a pass than pass following a pass. Therefore, offenses are not randomizing as much as they are alternating.

Play calls are not independent of the previous plays, even for similar down and distance situations. NFL offenses are therefore substantially, although not completely, predictable.

Instead of HTHHTHTH in a statistics class, we have RPRRPRPR in football. The patterns remind me of a language with consonants and vowels. But play calls are not simple either/or run or pass decisions. There are several variations to each, just like there are a's and e's and b's and c's.

Linear B was an ancient written language found in Crete and named for its straight lines. It was a precursor to the Hellenic Greek language dating back to the times of the Homeric epics. Several tablets with etchings in Linear B were excavated in Knossos, thought to be the capital of King Minos. The script was completely unlike any other, and baffled archeologists for decades. Researchers had nothing to go on except the patterns found in the writing.

In the mid-1940s Alice Kolber, an American professor, theorized that the characters represented syllables and that the language was highly inflective (having lots of different conjugations). Then in the 1950s, an amateur archeologist named Michael Ventris cracked the code. He found that each character represented a consonant-vowel combination. Each sound a person can make either goes well with others or it doesn't. This was all that was needed to eventually decode Linear B and unlock all its secrets.

Just like vowels and consonants, runs and passes tend to alternate. And certain types of plays tend to work well before and after others, just like "th" or "rn" or other consonant combinations.

I'm not claiming that we can crack the code on play-calling any more than you can predict my next word. My point is that a serious cryptographic analysis of play-calling could reveal tendencies not previously thought possible. For example, try to predict the next letter of the word "th..." Chances are very good it's a vowel, and if it's not, it's got to be an 'r.'

I'm sure coaches pore over hours of film trying to discern opponent tendencies, and are looking for things I couldn't even fathom. But it seems that they are focusing on situations in isolation. They're zeroing in on observations like, "they run on 2nd and long 35% of the time in the red zone." Apparently, coaches are not picking up on the fact that the same team might run 65% of the time in the same situation following a pass. If they were detecting these patterns, they wouldn't let their own offenses be so predictable.

I realize this is a wondering essay. My main points are that:

1. Offenses need to be unpredictable to be effective.
2. Plays need to be random both with respect to previous instances of the same down/distance situation and with respect to previous plays.
3. NFL offenses show evidence of patterns, even when holding for situational effects.
4. Coaches don't seem to be aware of the patterns, and they can be exploited.
5. And lastly, the only true countermeasure is to somehow inject genuine randomness into play calls.

I was going to finish this article by retelling an Edgar Allen Poe story in The Purloined Letter. But as I researched the details of the story, I realized I was beaten to the punch by the Smart Football blog. Check out this article on Poe, rock-paper-scissors, and play-calling.

Are Coaches Aggressive Enough on 2nd and 1?

In a recent post we saw that having 2nd down and 1 is actually preferable to a 10-yard gain for a 1st down. So if 2nd and 1 is really that valuable, are NFL offenses taking advantage when they get one? Are they actually taking shots down the field? The advantage is only as big as offenses make it. This post will look at how coaches exploit the 2nd and 1 situation.

Coaches are calling run plays on 78% of 2nd and 1 situations, which is even more than the 76% share for 3rd and 1s. For all 2nd down situations, runs are called 50% of the time. This indicates that no, coaches are not capitalizing on the opportunity and are treating the 2nd and 1 simply like an extra 3rd down.

Only 4% of 2nd and 1 plays are long pass attempts, defined as passes at least 15 yards down through the air. This is fewer than nearly all other 2nd down situations (only 2nd and 4 has fewer deep attempts). The remaining 18% are "short" pass plays, but those could be as long as 14 yards. It's these short passes that have generated most of the tactical advantage that makes 2nd and 1 preferable to a fresh 1st down. The chart below lists each play type and their associated expected points, the number of each type and the percentage.

Play Types on 2nd Down and 1

	Run	Short Pass	Deep Pass
Exp Pts	1.9	2.2	2.2?
Count	231	54	13*
Share	78%	18%	4%

Because there were only 13 deep pass attempts in the entire data set, I had to estimate their expected points from a larger set of plays. I grouped the deep passes on 2nd down and 1 through 4 yards to go. The resulting average expected points was 1.9, however, the expected points for 2nd and 1 would be higher. If unsuccessful, having a 3rd and 1 is far preferable to a 3rd and 2, 3, or 4 by a weighted average of 0.5 expected points. Deep passes are successful an average of 35% of the time, so this difference would be realized 65% of the time. This equates to a bonus of at least 0.3 expected points.

This suggests that the advantage in scoring from having a 2nd and 1 exists almost purely from the inherent nature of the 2nd and 1, and not due to coaches' deliberate efforts to capitalize on the opportunity. Imagine how large the advantage could be if coaches exploited it with more passes--short or long.

Although it would be hard (or impossible) to convince players to refrain from that second effort plunge at the 1st down marker, it may be easier selling coaches on taking greater advantage of the 2nd and 1 when they do come along.

2nd Down and 1

Al Michaels: “First and 10 from the 30. Campbell back to pass…it’s a screen to Portis. Right sideline…a 9-yard, no make that a 10 yard gain with the spot. It’ll be 1st and 10 for the Redskins at the 40.”

John Madden: “Yeah, Al. That’s just a totally Portis thing. He just knows where the first down marker is by instinct. See, right here [during the replay], he just --BAM!—reaches across the line for the first down. It’s like he’s got radar, Al.”

Actually, in a strange way, 'Sherriff Gonna Getcha' may have just unwittingly cost his team almost a point.

Portis may have cost his team almost a point (on average) because he passed up the juiciest down-and-distance situation of all: the 2nd and 1.

It must give defensive coordinators across the league nightmares. An offense can do anything on 2nd and 1. It can run, and probably pick up the first down, but they could just as easily take a shot down the field without much risk. The QB has the luxury of a no-pressure down. He doesn't have any need to force the pass and can throw it away if needed. An incompletion still leaves a very manageable 3rd and 1. And failing that, an attempt on 4th and 1 may not be out of the question (especially if it's a short 1).

A look at the expected points for 1st and 10 situations shows there is statistical evidence that a 9 yard gain is actually preferable than a 10-yard gain. The graph below shows the average expected points for most 1st and 10 plays in the 1st half of all regular season games from 2000 to 2007. (I excluded plays inside field goal range (the 35) and plays within 2 minutes of halftime. This limits the data to normal football situations, when teams are neither desperate nor nursing big leads, and when time is not a consideration. It also removes any bias in the data due to having the option to play very conservatively inside FG range.)


We can see a fairly clear drop in expected points from a 9-yard gain to a 10-yard gain from 2.3 points to 1.6. That's about a 0.7-point drop in the average number of points scored between having a 2nd and 1, and actually getting the first down. It may not sound like a lot of points, but it's a relatively large difference for a single yard on a single play.

There is, however, some noise in the data. So how can we be sure that the sudden discontinuity between 9- and 10-yard gains isn't just a very large random blip? First, the blip is fairly large. In fact, it's the largest jump between any two yardage gains. Second, it goes in the opposite direction we'd expect it to go. Further, the graph indicates that on 1st down, a 9-yard gain is not only better than a 10-yard gain, but it's better than anything up to a 16-yard gain. It also indicates that a 2nd and 3 is notionally just as good as converting 1st and 10. Lastly, we have a good theoretical basis as to why we'd see such a result.

So am I actually suggesting that ball carriers should intentionally try for a 9-yard gain instead of try for the extra 1 or 2 yards? It might be a hard sell, but yes, the evidence is there. On the other hand, the first time anyone actually did it intentionally, and his team failed to convert the 1st down, the criticism would be merciless and it would never be done again.

However, it appears that it may already be happening, at least unintentionally. The graph below plots the frequency of each gain (or loss) from a 1st and 10. Notice the the divot at exactly 10 yards. There is an unnaturally low number of 10-yard gains compared to 9- and 11-yard gains. This could be due to how refs spot the ball or how defenses guard the 10-yard marker, but it's intriguing.


So if 2nd and 1 is really that valuable, are NFL offenses taking advantage when they get one? Are they actually taking shots down the field? After all, the advantage is only as big as offenses make it. Perhaps it could be even larger if coaches properly exploited the situation. I'll take a look at that in the next post.

Edit: I hope no one thinks I'm suggesting untouched ball carriers should spontaneously drop after 9 yards. I'm only suggesting that the outstretched arm/second effort thing can be strangely counterproductive. But mostly I'm just illustrating how the rules of football sometimes create counter-intuitive effects.

Play Calling on 3rd and Short Part 3

In this third and final installment looking at play calling on third down, I'll analyze the statistical relationship between individual team's tendencies and their conversion rate. So far, in part 1 and part 2, we've seen that running on 3rd and short tends to get more first downs and leads to scoring more points. So, are individual teams that tend to run on third and short more successful than teams that tend to pass?

A Case Study

One of the notable differences between Patriots’ coach Bill Belichick and most other coaches is his tendency to run on 3rd and short. I cannot recall where I read that, and I’ve wanted to examine 1—if its true, and 2—how it affects his team’s success on third down.

First, yes, it is true. In his tenure with the Patriots, Belichick has run 80% of the time on 3rd and 1 compared to a league-wide average of 70%. This rate makes the Pats tied for 3rd in the NFL for run tendency in that time frame. Their overall conversion rate for all 3rd down and 1 situations was 5th in the league at 72%. So Belichick’s reputation is at least partially true. His teams are near the top of the league in both categories, but not at the very top.

But what about the rest of the teams? If there is a statistical link between run tendency and conversion rate across the NFL, this would be confirmation that teams should run more often on 3rd and short.

The correlation between percentage of run attempts and conversion rate on 3rd and 1 situations is 0.19 (which is statistically significant at p<0.01, n=254.) This seems pretty big considering all the factors that go into conversion rate—team strength, opponent strength, luck. Simply calling more run plays would significantly improve most teams’ conversion rates on 3rd and short. The effect may be somewhat overstated, however. Not all 3rd and 1s are equal. Both 3rd and 1 inch, and 3rd and 1½ yards are considered ‘3rd and 1’ according to the NFL. So teams that have a disproportionate number of 3rd and 1 inch situations would logically run more often, and more successfully. But any bias in the correlation would likely be very small as there are over 28,000 3rd and 1 plays in the data set, and any effect should even out among teams. Additionally, 3rd and 2 and 3rd and 3 situations do not exhibit that complication, but the evidence that teams pass too often is just as strong if not stronger.

The total of all the evidence makes the conclusion clear. Offenses should run more often on 3rd and short. Based on conversion rate, expected points, and team-by-team correlation, running more often on third and short leads to more 1st downs, more points, and consequently more wins.

Play Calling on 3rd and Short Part 2

Are NFL coaches calling the right plays on 3rd down and short? In part 1, we saw that 3rd down conversion rates suggested teams should run more often in such situations, as long as getting a first down is the only goal. In this installment, I'll continue the analysis by considering two other factors--average yardage gained and expected points.

Adjusted Gain

Although conversion rates indicate teams pass too often, they're not the only measure of success on 3rd down. It’s always better to have a 1st down and more yards (except on 3rd and goal). So on one hand, passing seems to offer an advantage over running in that its average gain is longer. On the other hand, there is the chance of an interception when passing. To include both considerations in the analysis, I’ve calculated the “adjusted gain” of each play. Adjusted gain subtracts 45 yards for each interception. This method accounts for interception risk well because a 45 yard loss results in roughly the same change in expected scoring as a turnover. The graph below plots the distribution of adjusted gain for 3rd and 1 situations.


A simpler way of looking at each option is the average adjusted gain for each type of play. Here is a graph of average adjusted gain for 3rd down situations.


Until 3rd and 4 offenses are getting more adjusted yards by passing than by running. By 4 yards to go, adjusted gain equalizes. (Again, except for 3rd and short, it’s striking just how equal the adjusted gains are.) On 3rd and 1 the pass outgains the run by 3.3 yards per play, but the run gets a first down more often. The question now becomes, what would an offense rather have: an extra 12% probability of conversion by running, or an extra 3.3 yards on average by passing?

Expected Points

By comparing the expected points of both options, we can get an answer. A typical punt results in a loss of about 1.8 expected points. This is 12% more likely when passing than when running, which results in a 0.12 * 1.8 = 0.22 loss in expected points. An extra 3.3 yards is worth 0.10 additional expected points. The net difference is 0.12 points in favor of the run. So for 3rd and 1, the extra gain achieved by passing is not worth the added risk of the failing to convert.

The 0.10 expected points from passing’s added gain would be worth it only when running is successful 5.5% more often than passing. This would be the true Nash equilibrium for 3rd and 1.

x * 1.8 = 0.10
x = 0.10/1.8
x = 0.055 = 5.5%

But after I did that little algrebra exercise, I realized it was unnecessary because I already have the historical data for actual expected points for each situation. The graph below shows how many points, on average, a team scored following a 3rd down. Expected points after a run or pass shows that offenses pass too often on 3rd and short and run too often on 3rd and long.


On 3rd and 1, offenses scored an average of 2.38 points if they ran, and an average of 2.24 points if they passed. The difference of 0.14 points is remarkably close to the 0.12-point theoretical estimate calculated above.

The differences are even bigger for 3rd and 2 and 3rd and 3 situations. The advantage of running is 0.45 and 0.31 expected points respectively—convincing evidence that offenses should be running more often on 3rd and short. The graph also indicates passing is ultimately more fruitful on 3rd and long, despite equal conversion rates.

In the third and final part of this article, I'll look at whether teams that run more often on third and short really are more successful. I'll also quantify just how important optimum play calling can be.